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RESUME 


On  d£cnt  un  systfeme  interactif  de  traitement  num^rique  des 
images.  Ce  systfcme  comprend  un  mim-ordinateur ,  un  processeur 
programmable  de  tableau  et  de  l'gquipement  en  direct  servant  &  la 
digitalisation  et  h  l'affichaqe  des  images.  L 'utilisation  de  ce  systfeme 
pour  la  reconstitution  et  1 ' amelioration  des  images  est  expliqu^e.  On 
analyse  une  photographie  floue  prise  avec  une  camera  hors  de 
localisation  pour  determiner  la  nature  du  flou,  puis  on  reconstitue 
1' image.  On  proc&de  6galement  &  1 1  amelioration  des  images  de  types 
spectre  visible,  infrarouge,  et  radiographique  en  utilisant  des 
techniques  fondees  sur  des  caracteristiques  du  systfeme  visuel  humain  et 
sur  le  processus  de  formation  des  images.  (NC) 


'TV'S  ABSTRACT 

'  v  - 

describe /  an  interactive  digital  image-processing  system  that 
includes  a  minicomputer,  a  programmable  array  transform  processor  and 
on-line  image  digitizing  and  display  equipment,  and  illustrate  typical 
applications  in  the  areas  of  image  restoration  and  enhancement.  A 
photograph  blurred  by  a  defocused  camera  is  analyzed  to  determine  the 
nature  of  the  blur  and  is  restored.  Image-enhancement  techniques,  based 
on  the  properties  of  human  vision  or  on  the  image- format ion  process,  are 
applied  to  visible  light,  infrared  and  radiographic  images.  (U) 
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1.0  INTRODUCTION 


Images  that  are  defective  or  deficient  in  some  respect  can  often 
be  improved  through  processing  with  a  digital  computer.  The  purpose  of 
the  processing  may  be  to  remove  a  specific  degradation,  motion  or  focus 
blurs  for  example,  or  to  enhance  information  of  interest.  In  general, 
current  automatic  image- improvement  techniques  (e.g.  Refs.  1  and  2) 
cannot  yet  equal  what  a  human  operator  can  achieve  interactively. 

Interactive  processing  puts  the  man  "in  the  loop"  where  he  is 
able  to  rapidly  inspect  the  processed  image  and  apply  new  processing 
based  on  his  judgement  and  experience.  The  recent  development  of 
digital  hardware  that  can  efficiently  process  the  large  amount  of 
information  present  in  a  typical  high-resolution  image,  now  makes  this 
processing  practical  with  a  minicomputer-based  system. 

In  this  report,  we  describe  an  interactive  digital 
image-processing  system  based  around  a  PDP  11/40  computer  and  illustrate 
its  processing  capabilities  by  giving  examples  of  interactive  image 
restoration  and  enhancement.  The  system  is  not  limited  to  its  present 
role  of  image  improvement  (some  examples  are  given  in  Refs.  3-7)  but  is 
intended  as  a  flexible  research  facility  for  investigating  2-dimensional 
signal-processing  and  display  techniques.  Other  current  applications 
include  a  real-time  simulation  of  a  target  correlation  tracking  system 
(Ref.  8)  and  a  study  of  automatic  target  acquisition  (Refs.  9-11).  In 
general,  the  system  permits  potential  real-time  image-processing 
techniques,  including  those  which  require  less  operator  supervision,  to 
be  evaluated  or  optimized  by  simulation  before  their  final  hardware 
implementations . 

The  experimental  system  described  in  Sec.  2.0  includes  on-line 
equipment  for  digitizing  film,  for  real-time  TV  display  and  for 
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recording  on  Polaroid  film  images  containing  up  to  512  by  512  elements. 
A  special-purpose  hardware  processor  interfaced  to  the  host  computer 
allows  complex  processing  operations,  such  as  2-dimensional 
frequency-domain  filtering,  to  be  performed  on  256-  by  256-element 
images  in  several  seconds.  In  Sec.  3.0,  we  give  an  example  of 
interactive  restoration  of  a  blurred  photograph  by  inverse  Wiener 
filtering  and  in  Sec.  4.0,  we  show  several  examples  of  interactive  image 
enhancement. 

Ibis  work  was  performed  at  DREV  during  1977-78  under  PCN  33A40, 
Digital  Image  Processing  and  PCN  21J03,  Imaging  Seekers. 
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2.0  EXPERIMENTAL  SYSTEM 

Effective  interactive  digital  image  processing  requires  rapid 
processing,  flexible  methods  for  controlling  the  various  operations,  and 
convenient  facilities  for  viewing  and  permanently  recording  the  results. 
The  filtering  or  display  operations  that  we  use  most  frequently  should 
be  completed,  and  the  results  viewed,  within  a  few  seconds. 

Our  interactive  system  is  based  on  a  PDP  11/40  computer  with  24 
kwords  of  core  memory.  Image  processing  and  image  input  or  output  (I/O) 
operations  are  controlled  through  commands  given  on  a  teletype.  A  brief 
description  of  the  image-processing  operating  system  is  given  in 
Appendix  A  while  a  listing  of  the  interactive  commands  currently 
available  is  given  in  Appendix  B.  Options,  such  as  the  size  of  the 
image  to  be  processed  (128  by  128,  256  by  256  or  512  by  512  elements) , 
or  that  of  the  display  area  (128  by  128,  256  by  256  or  512  by  512 
elements) ,  are  selected  by  setting  sense  switches  on  the  computer 
console.  Film  transparencies  or  prints  are  digitized  with  a 
high-quality  vidicon  scanner  (I1M  Model  201/E)  interfaced  to  the  PDP 
11/40  computer  through  a  video  digitizer  (CVI  Model  270) .  An  analog 
scan  conversion  memory  (PEP  Model  400)  driving  a  1225-line  TV  monitor 
provides  a  real-time  display.  Images  are  recorded  on  film  with  a 
flying-spot  scanner  (Tektronix  Model  602  or  606)  viewed  by  a  Polaroid 
camera.  The  system  is  linked  to  a  Honeywell  560  digital  computer,  an 
EAI  8845  hybrid  computer  and  an  Optronics  P-1700  rotating-drum 
microdensitometer/film  writer  through  a  9-track  magnetic  tape. 

A  programmable  array  transform  processor  (Floating  Point  Systems 
Model  AP120-B)  interfaced  to  the  PDP  11/40  computer  is  a  key  element  of 
the  system.  It  has  decreased  the  computing  time  required  for  typical 
image-processing  operations  by  a  factor  of  approximately  300,  allowing 
complex  filtering  operations,  which  previously  required  a  medium-scale 
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digital  computer,  to  be  performed  interactively.  For  example,  the 
2-dimensional  fast  Fourier  transform  (2D  FFT)  of  a  256-  by  256-element 
image  is  performed  in  1.4  seconds. 

One  usually  uses  an  interactive  system  to  process  images  with  a 
spatial  resolution  of  512  by  512  elements  or  less.  In  such  cases,  a 
vidicon  camera  designed  for  high  gray  scale  and  geometric  accuracy  can 
be  an  acceptable  alternative  to  a  slower  and  more  costly 
microdensitometer  for  digitizing  film.  In  a  vidicon  system,  we  select 
the  region  of  the  film  to  be  digitized  by  adjusting  the  camera  optics 
and  film  position  while  we  view  the  camera  output  on  a  TV  monitor. 

Provided  at  least  10  TV  frames  are  averaged  to  sufficiently 
reduce  vidicon  noise,  the  camera  yields  a  gray  scale  resolution  of  0.03D 
over  the  density  range  0-3D.  Noise  due  to  film  granularity  exceeds  this 
level  at  most  effective  sampling  apertures  and  we  have  found  this 
resolution  adequate  for  many  deblurring  or  enhancement  applications. 
The  main  disadvantages  of  the  vidicon  digitizing  system  are  its  limited 
number  of  spatial-resolution  elements,  the  long-term  image  retention  by 
the  vidicon  target  and  the  need  to  calibrate  the  spatial  and  density 
scales.  On  the  other  hand,  unlike  most  microdensitometers,  the  optics 
of  the  camera  lens  and  the  illumination  system  can  be  adjusted  to  allow 
measurement  of  specular  or  diffuse  film  density.  Measurement  of  the 
specular  density  of  a  silver  halide  film  produces  a  sharper  image  than 
measurement  of  diffuse  density  but  at  the  expense  of  a  higher  level  of 
film  noise.  When  noise  due  to  film  granularity  is  the  limiting  factor 
(as  is  frequently  the  case  in  image  deblurring  or  enhancing  of 
low-contrast  details,  for  example) ,  measurement  of  diffuse  density  may 
be  preferred,  whereas  specular  density  may  be  preferred  for  optimum 
enlargement  of  small  details. 


UNCLASSIFIED 

5 


The  array  transform  processor  processes  the  images  under  control 
of  the  host  computer.  In  a  typical  filtering  operation,  the  image  (or 
one  quarter  of  it,  in  the  case  of  a  512-  by  512-element  image)  is 
transferred  from  an  RK05  disk  memory  to  the  65  kword  data  memory  of  the 
array  transform  processor.  High-speed  disk  I/O  routines  (Ref. 12),  which 
are  approximately  10  times  faster  than  the  standard  Fortran  I/O 
routines,  were  written  and  these  allow  65  thousand  16-bit  words  to  be 
transferred  between  a  disk  and  the  array  processor  data  memory  in  1.4 
seconds.  The  image  is  then  processed  at  a  rate  of  up  to  12  million 
operations  per  second  according  to  algorithms  which  the  host  computer 
has  placed  in  the  program  memory  of  the  array  transform  processor.  The 
processed  image  is  scaled  and  returned  to  a  disk  or  displayed.  A  256- 
by  256-element  image  is  displayed  or  recorded  on  film  in  2.5  seconds. 

We  plan  to  make  several  improvements  in  the  system.  The  gray 
scale  resolution  of  the  analog  scan  conversion  display  memory  is  limited 
and  degrades  after  about  15  min  of  readout.  This  will  be  replaced  by  a 
digital  color  display  system  with  graphics  and  interactive  capabilities. 
A  time-shared  PDP  11/70  computer  will  be  added  to  the  system. 
Low-priority  tasks,  such  as  editing  or  linking  programs,  will  be 
performed  at  the  same  time  that  a  "privileged"  user,  who  controls  the 
array  transform  processor  and  on-line  image  I/O  equipment,  operates  in 
real  time. 
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3.0  IMAGE  RESTORATION 

Itie  objective  of  image  restoration  is  to  remove  a  specific 
degradation  and  restore,  as  closely  as  possible,  to  some  "ideal"  image. 
The  degradation  may  be  caused  by  the  imaging  process  itself  (e.g. 
transmission  through  atmospheric  turbulence) ,  by  the  imaging  system 
(e.g.  an  improperly  focused  camera)  or  by  the  image-recording  medium 
(e.g.  nonlinearity  of  photographic  film).  Typical  types  of  image 
degradations  that  one  can  restore  include  blurring  (e.g.  due  to  motion 
or  focus),  geometrical  distortion  (e.g.  due  to  perspective)  and  gray 
scale  nonlinearity  (e.g.  due  to  film  overexposure  or  underexposure) .  In 
this  section,  we  illustrate  the  use  of  interactive  image  processing  to 
identify  and  remove  focus  blur  from  a  photograph. 

Figure  1(a)  shows  an  image,  recorded  on  Kodak  Panatomic  X  film, 
which  was  blurred  in  a  camera.  We  first  analyze  the  image  to  determine 
the  type  of  blurring  (motion,  focus,  etc.)  and  its  extent.  This  can  be 
described  by  the  optical  transfer  function  (OTF)  of  the  blur.  Gennery 
(Ref.  13)  has  shown  that  the  modulation  transfer  function  (MTF) ,  i.e. 
the  modulus  of  the  OTF,  of  a  spatially  invariant  blur  can  frequently  be 
observed  in  the  power  spectrum  of  the  blurred  image.  If  the  MTF  pattern 
contains  characteristic  structures  (e.g.  zeros)  which  are  sufficiently 
visible  in  the  power  spectrum,  then  this  can  help  to  determine  the  OTF 
of  the  blur.  The  ring  structure  in  the  power  spectrum  of  the  present 
image  (Fig  .  1(b))  is  typical  of  blurring  due  to  an  out-of-focus  camera 
with  a  circular  aperture.  The  OTF  of  this  type  of  blur  is  a  first-order 
Bessel  function  (Refs.  1  and  14) ,  and  the  2  minima  observed  in  the  power 
spectrum  correspond  to  the  first  2  zeros  of  the  Bessel  function.  Hie 
positions  of  the  minima  estimate  the  blur  circle  to  have  an  average 
diameter  of  0.68  im  on  the  original  film  negative. 
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FIGURE  1  -  Image  deblurring.  A  region  of  a  photograph  blurred  in  a 
defocused  camera  is  shown  in  (a) .  Its  power  spectrum  is 
given  in  (b) .  In  (c)  we  have  restored  the  blur  by  inverse 
Wiener  filtering.  The  same  region  of  the  sign  taken  with  the 
camera  correctly  focused  is  given  in  (d) . 
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Before  performing  the  restoration,  we  first  match  the  gray  levels 
of  opposite  edges  over  a  3-element  wide  region  bordering  the  image  to 
reduce  artifact  generation  (Refs.  6,  7  and  15).  Then,  we  calculate  the 
2D  FFT  of  the  image,  multiply  by  the  inverse  Wiener  filter  (Refs.  7  and 
14)  generated  using  the  above  Bessel-function  OTF,  and  calculate  the 
inverse  2D  FFT.  The  resulting  restored  image  (Fig.  1(c))  is  much  less 
blurred  than  the  original;  text  which  is  illegible  in  the  original  image 
can  easily  be  read  after  restoration.  Figure  1(d)  shows  the  same  region 
of  the  sign  taken  with  the  camera  correctly  focused  for  comparison. 

Noise  due  to  film  granularity  and,  to  a  lesser  extent,  spatial 
variance  of  the  blur,  limited  the  amount  of  restoration  that  could  be 
achieved  in  Fig.  1.  (Power-spectrum  analysis  of  smaller  regions 
selected  from  the  complete  image  shows  that  the  upper  portion  of  the 
image  is  defocused  by  about  4%  less  than  is  the  lower  portion.)  When 
the  blurring  varies  significantly  from  one  part  of  the  image  to  another, 
one  can  often  select  and  separately  restore  smaller  regions  of  the 
complete  image  over  which  the  blur  is  sufficiently  constant.  In  a  few 
special  cases,  such  as  coma  aberration  of  a  spherical  lens  (Ref.  14),  we 
can  remove  a  spatially  variant  blur  by  applying  a  geometrical 
transformation  to  the  image  to  make  the  blur  spatially  invariant,  by 
restoring  this  image  and  by  applying  the  inverse  geometrical 
transformation. 

Power-spectrum  analysis  will  not  always  give  sufficient 
information  to  determine  the  OTF  of  the  blur.  The  MTF  pattern  may  not 
be  distinctive  enough  (for  example,  the  MTF  of  blurring  due  to 
time-averaged  atmospheric  turbulence  is  almost  gaussian) ,  or  it  may  be 
hidden  by  noise  or  by  structure  in  the  power  spectrum  of  the  original 
(unblurred)  image.  If  we  can  correctly  estimate  the  general  type  of 
blurring,  by  a  priori  knowledge  or  by  inspection  of  the  blurring  of 
edges  or  points  in  the  image  for  example,  then  its  extent  may  be 
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obtained  as  illustrated  in  Fig.  2.  Here,  we  assumed  that  the  image  was 
blurred  in  a  defocused  camera  with  a  circular  aperture.  Then,  we 
restored  a  128-  by  128-element  region  of  the  blurred  image  shown  in  Fig. 
1(a)  by  using  16  values  of  assumed  blur -circle  diameter  ranging  from  0 
mm  (no  restoration)  to  0.75  tim  (too  much  restoration)  in  0.05  mn  steps. 
An  operator  who  views  such  a  multiple-image  display  can  choose  the  best 
restoration  and,  assuming  spatially  invariant  blurring,  restore  the 
complete  image  in  the  same  way.  In  this  case,  it  is  evident  that  the 
best  restoration  occurs  for  0.65-mm  blur  circle  diameter,  in  agreement 
with  the  power-spectrum  analysis  procedure.  Only  55  seconds  real  time 
was  required  for  the  system  to  perform  the  16  restorations  and  to 
display  the  results  on  the  TV  monitor. 

We  performed  the  above  Wiener-filter  restorations  assuming  that 
the  original  image  had  a  constant  signal-to-noise  ratio  equal  to  30  dB. 
(The  choice  of  a  constant  signal-to-noise  ratio,  rather  than  one  which 
varies  with  frequency,  has  been  discussed  in  Ref.  7) .  The  use  of  a 
higher  value  of  signal-to-noise  ratio  produces  more  restoration  but  at 
the  expense  of  a  noisier  image  and  more  severe  artifact  generation  (e.g. 
due  to  film  nonlinearity,  a  spatially  variant  blur,  dust  or  scratches  on 
the  film  etc.).  In  practice,  it  appears  preferable  to  choose  this  value 
subjectively  rather  than  to  use  the  true  average  signal-to-noise  ratio 
of  the  image.  The  signal-to-noise  ratio  of  photographic  film  varies 
with  film  exposure,  so  that  no  single  value  will  be  correct  for  all 
regions  of  an  image  (Ref.  7) .  Furthermore,  we  often  want  to  optimize 
the  restoration  of  a  particular  feature,  rather  than  the  complete  image, 
and  this  is  best  achieved  interactively  by  trial  and  error. 
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FIGURE  2  -  Determination  of  the  OTF  of  a  blur  interactively.  A  region 
of  a  focus-blurred  image  was  restored  assuming  16  different 
blur-circle  diameters.  The  values  start  at  0  mm  (i.e.  no 
restoration)  (upper  left)  and  increase  in  0.05-mm  steps  to 
0.75  mm  (lower  right).  A  blur-circle  diameter  of  0.65  mm 
gives  the  subjectively  best  restoration. 
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4.0  IMAGE  ENHANCEMENT 


4.1  Ad-Hoc  Techniques 


Image  enhancement  is  harder  to  define  than  image  restoration; 
there  is  usually  no  "ideal  image"  which  the  processing  tries  to  achieve. 
Andrews  (Ref.  16)  has  defined  enhancement  as  the  attempt  to  improve  the 
appearance  of  an  image  for  human  viewing  or  subsequent  machine 
processing. 

The  choice  of  an  enhancement  technique  is  simplified  if  the 
information  of  interest  differs  in  some  significant  respect  from  the 
remainder  of  information  in  the  image.  Figure  3(a),  for  example,  shows 
an  aerial  photograph  of  a  flock  of  birds  flying  over  water.  We  want  to 
extract  the  birds  (the  numerous  small  white  objects) ,  so  that  a 
subsequent  computer  algorithm  can  count  and  analyze  them.  The  spatial 
and  gray  scale  characteristics  of  the  birds  are  significantly  different 
from  those  of  the  background.  We  first  enhance  the  birds  by  high-pass 
filtering  and  then  eliminate  the  background  by  setting  any  image  element 
with  a  gray  level  below  a  fixed  threshold  to  full  black  (Fig.  3(b)). 

In  the  ballistic-range  photograph  shown  in  Fig.  3(c),  the 
objective  of  the  enhancement  is  to  extract  the  horizontal  and  vertical 
reference  lines  and  the  projectile,  so  that  a  computer  algorithm  can 
measure  the  relative  position  and  orientation  of  the  projectile.  We 
first  apply  a  directional  filter  to  select  all  horizontal  and  vertical 
lines  and  edges  in  the  image.  This  eliminates  the  diagonal  lines  caused 
by  shock  waves  and  strongly  attenuates  other  features.  After  high-pass 
filtering  to  enhance  the  sharp  horizontal  and  vertical  reference  lines 
relative  to  the  "softer"  horizontal  edges  of  the  projectile  and  its 
shadow,  we  detect  the  reference  lines  by  setting  all  image  elements  with 
a  gray  level  exceeding  a  fixed  threshold  full  white,  and  all  others  full 
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(c)  (d) 


FIGURE  3  -  Ad -hoc  image  enhancement.  The  two  original  images  show  an 
aerial  photograph  of  a  flock  of  birds  (a) ,  and  a  photograph 
of  an  artillery  shell  in  a  ballistic  range  (c) .  In  both 
cases,  the  spatial  and  grayscale  characteristics  of  the 
information  of  interest  are  significantly  different  from 
those  of  the  background.  This  information  can  easily  be 
enhanced  by  appropriate  filtering,  (b)  and  (d) . 


UNCLASSIFIED 

13 


black.  We  similarly  low-pass  filter  the  original  image  to  attenuate  the 
reference  and  shock-wave  lines  and  then  apply  threshold  detection  to 
produce  an  image  which  contains  only  the  projectile.  Figure  3(d)  shows 
the  result  of  adding  the  image  containing  the  detected  reference  lines 
to  the  image  containing  the  detected  projectile. 

The  above  two  examples  took  advantage  of  distinctive  spatial  and 
gray  scale  properties  of  the  information  of  interest  as  they  appeared  in 
the  final  film  images.  A-priori  knowledge  of  these  properties,  combined 
with  interactive  determination  of  the  appropriate  filtering  procedures, 
allowed  us  to  enhance  the  information  of  interest.  In  general,  however, 
the  images  will  be  more  complex  than  either  of  the  above  examples,  and 
the  information  to  be  enhanced  will  not  be  as  well  differentiated  from 
the  background.  In  such  cases,  we  should  use  all  the  information 
available  in  performing  the  enhancement.  This  includes  all  knowledge  we 
have  of  the  characteristics  of  the  image- formation  process,  the  imaging 
sensor,  the  image-recording  medium  and,  if  the  image  is  intended  for 
human  interpretation,  the  display  system  and  the  properties  of  the  human 
visual  system. 

A  unified  approach  to  image  enhancement  which  includes  these,  and 
possible  other  significant  factors,  has  yet  to  be  established.  Most 
enhancement  techniques,  such  as  those  illustrated  in  Fig.  3,  are  ad  hoc 
and  are  not  based  on  how  the  image  to  be  enhanced  was  produced.  We  next 
illustrate  how  two  of  the  above  factors,  considered  separately,  can  be 
applied  to  image  enhancement.  Section  4.2  shows  how  the  properties  of 
the  image-formation  process  can  enter  into  the  enhancement  problem, 
while  Sec.  4.3  describes  an  enhancement  technique  based  on  the 
characteristics  of  human  vision. 
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4.2  Enhancement  Based  on  the  Image  Formation  Process 

Orvenheim  et  al  (Ref.  17)  have  described  homomorphic  filtering  as 
a  technique  for  separating  signals  into  additive  parts  and  enhancing  the 
parts  of  interest  compared  to  those  of  less  interest.  They  showed  how 
a  visible-light  image,  which  is  assumed  to  be  formed  by  multiplying  an 
illumination  part  by  a  part  due  to  target  reflectivity,  can  be  enhanced 
by  reducing  the  contrast  caused  by  changes  in  illumination  and 
increasing  the  contrast  due  to  changes  in  reflectivity.  In  their 
example,  the  homomorphic  filtering  consisted  in  calculating  the 
logarithm  of  the  image  to  make  the  illumination  and  reflectivity  parts 
additive,  high-pass  filtering  to  attenuate  the  low-frequency 
illumination  part  relative  to  the  higher  frequency  reflectivity  part, 
and  then  exponentiating.  As  we  illustrate  in  the  next  two  paragraphs,  a 
similar  type  of  homomorphic  filtering  may  be  used  to  enhance  images 
formed  by  the  emission  of  infrared  radiation,  as  well  as  certain  kinds 
of  images  formed  by  the  transmission  of  X  or  gamma  radiation. 

An  image  formed  by  the  emission  of  infrared  radiation  consists  of 
a  part  which  depends  on  target  temperature  multiplied  by  the  emissivity 
of  the  surface.  Because  of  thermal  conduction  within  the  target,  the 
temperature-dependant  part  may  show  a  less  sharp  image  of  the  target 
than  the  emissivity  part.  For  targets  at  close  range  where  this  limits 
the  resolution,  we  may  be  able  to  sharpen  the  target  image  by  using 
multiplicative  homomorphic  filtering  to  attenuate  the  lower  frequency 
temperature-dependant  part.  Figures  4(a)  and  (b)  show  the  result  of 
applying  this  filter  to  an  infrared  image  formed  in  the  9-  to  13-ym 
band.  As  well  as  sharpening  the  image,  the  filtering  has  compressed  the 
dynamic  range  by  reducing  contrast  due  to  both  changes  in  target 
temperature  and  surface  emissivity  which  vary  slowly  from  one  region  of 
the  image  to  another. 
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FIGURE  4  -  Image  enhancement  based  on  a  knowledge  of  how  the  image  was 
formed.  Figures  (b)  and  (d)  show,  respectively,  compression 
of  the  dynamic  range  of  an  image  formed  by  emission  of 
infrared  radiation  (a) ,  and  an  image  formed  by  transmission 
of  gamma  radiation  (c) . 
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Images  formed  by  the  exponential  absorption  of  radiation  may  also 
require  multiplicative  homomorphic  filtering.  Suppose  we  take  separate 
radiographs  of  two  objects  (or  of  two  slices  of  the  same  object) .  If 
the  two  objects  are  superimposed,  and  a  third  radiograph  is  taken,  the 
result  will  be  the  product,  not  the  sum,  of  the  individual  radiographs. 
Consider  the  simple  case  where  the  first  object  contains  a  single  low 
spatial-frequency  component,  the  second  contains  a  single  high 
spatial-frequency  component,  and  we  want  to  filter  the  composite 
radiograph  to  remove  one  of  the  frequency  components.  Unless  we  first 
calculate  the  logarithm  to  make  the  two  frequency  components  additive, 
linear  filtering  may  not  be  able  to  remove  the  sum  and  difference 
frequencies  produced  when  the  two  frequency  components  are  multiplied 
(Ref.  6).  Figures  4(c)  and  (d)  illustrate  the  use  of  multiplicative 
high-pass  homomorphic  filtering  to  enhance  and  to  compress  the  dynamic 
range  of  a  gamma-ray  radiograph  of  an  artillery  shell.  The  filtering 
reduces  the  contrast  due  to  changes  in  the  thickness  of  the  shell,  which 
covers. a  wide  dynamic  range  but  presents  little  interest  here,  and 
enhances  higher  frequencies  corresponding  to  cracks  or  defects. 

4.3  Enhancement  Based  on  the  Properties  of  Human  Vision 

High-pass  spatial  filtering,  such  as  that  described  in  Sec.  4.2, 
is  an  effective  way  to  compress  the  dynamic  range  of  an  image  and  to 
sharpen  or  enhance  low-contrast  details.  This  processing  can  preserve 
information  that  might  otherwise  be  lost  due  to  an  inadequate  display 
medium  (e.g.  a  TV  monitor,  a  halftone  printing  process,  etc.).  However, 
it  often  introduces  artifacts  which  are  easy  to  misinterpret.  We  cannot 
be  sure  if  what  we  see  in  the  filtered  image  is  true,  or  if  it  was 
introduced  by  the  processing. 

The  hunan  eye  compresses  the  dynamic  range  of  visual  information 
by  high-pass  filtering  before  sending  it  along  the  optic  nerve  to  the 
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brain.  It  is  reasonable  to  assume  that  the  brain  decodes  this 
information  to  attempt  to  restore  to  the  original  image.  Certain  visual 
illusions,  such  as  the  Cornsweet  illusion  (Ref.  18) ,  suggest  that  this 
indeed  happens.  The  basis  of  the  present  enhancement  is  to  filter  the 
image  in  the  same  way  as  the  eye  performs  dynamic  range  compression.  At 
best,  the  brain  may  be  able  to  partially  restore  to  the  original  image. 
At  worst,  the  brain  may  be  less  likely  to  misinterpret  artifacts 
generated  by  this  filtering  than  those  generated  by  other  types  of 
high-pass  filtering;  the  same  artifacts  will  be  generated  later  by  the 
eye  anyway. 

Figure  5  shows  the  result  of  enhancing  an  image  using  the 
nonlinear  model  for  the  eye  proposed  by  Hall  and  Hall  (Ref.  19)  .  In 
this  model,  the  image  is  assumed  to  be  low-pass  filtered  by  the  lens  of 
the  eye,  logarithmically  detected  by  receptors  in  the  retina,  and  then 
high-pass  filtered  by  lateral  inhibition  between  neighboring  receptors. 
The  two  enhanced  images  shown  in  Figs.  5(b)  and  (c)  correspond, 
respectively,  to  low-frequency  responses  measured  by  Davidson  (Ref.  20) 
for  the  visual  systems  of  subjects  BM  and  LF.  The  frequency  scaling  is 
such  that  the  7 -an  square  processed  images  should  be  viewed  at  a 
distance  of  approximately  25  cm.  Images  that  have  been  enhanced  in  this 
way  are  similar  in  overall  appearance  to  the  original,  but  the  contrast 
in  both  light  and  dark  areas  has  been  increased  and  high-frequency 
details  have  been  sharpened. 
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FIGURE  5 


Image  enhancement  based  on  the  properties  Of  human  vision. 
Figures  (b)  and  (c)  illustrate  compression  of  the  dynamic 
range  of  the  original  image  (a)  by  filtering  according  to  a 
simple  model  for  the  human  visual  system.  The  two  enhanced 
images  correspond  to  the  frequency  responses  of  the  visual 
systems  of  two  individuals. 
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5.0  CONCLUSIONS 

We  have  described  an  interactive  digital  image-processing  system 
based  on  a  PDP  11/40  minicomputer.  It  includes  on-line  facilities  for 
digitizing  film,  for  TV  display  and  for  recording  images  on  film.  An 
array  transform  processor  interfaced  to  the  minicomputer  allows  complex 
processing  operations,  such  as  2D  FFT  filtering,  to  be  interactively 
applied  in  several  seconds  to  images  containing  256  by  256  elements. 

Such  a  system  has  numerous  applications  in  the  general  area  of 
2-dimensional  signal  processing.  We  have  outlined  2  current 
applications  involving  the  improvement  of  images  intended  for  human 
interpretation,  in  particular,  interactive  image  deblurring  and 
enhancement.  Interactive  processing  allows  us  to  optimize  the 
restoration  or  enhancement  of  features  of  special  interest  in  the  image, 
whereas  normally  the  processing  is  suited  to  the  average  properties  of 
the  image.  In  addition,  it  often  permits  easier  detection  of  artifacts 
generated  by  the  processing,  and  gives  us  a  better  overall  understanding 
of  what  effect  the  processing  has  on  the  image. 

The  deblurring  was  achieved  by  analyzing  the  image  to  identify 
the  type  and  the  extent  of  degradation,  and  then  restoring  the  image  by 
inverse  Wiener  filtering.  We  gave  an  example  of  interactive  analysis 
and  restoration  of  a  photograph  blurred  by  an  out-of-focus  camera.  The 
enhancement  was  achieved  in  3  ways:  through  ad-hoc  processing  ,  by 
taking  account  of  the  image-formation  process,  or  by  including  the 
properties  of  human  vision.  We  gave  4  examples  illustrating  the 
enhancement  of  images  formed  by  the  reflection  of  visible  light, 
emission  of  infrared  radiation  and  the  transmission  of  gamma  radiation. 
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APPENDIX  A 


Description  of  the  Image-Processing  Operating  System 

In  this  appendix  we  briefly  describe  how  a  user  controls  the  DREV 
image-processing  system  by  giving  interactive  commands  on  a  teletype  and 
by  setting  switch-selected  options. 

The  first  step  in  using  the  system  is  to  select  the  desired 
processing  options  by  setting  bits  on  the  switch  register  of  the  PDP 
11/40  console.  If  the  switch  register  is  not  set,  then  the  operator 
must  provide  this  information  by  replying  to  questions  on  the  teletype 
whenever  a  processing  operation  that  uses  these  options  is  executed. 

The  main  options  to  be  selected  are  the  size  of  the  working  image 
and  that  of  the  display  area.  The  size  of  the  working  image  represents 
the  size  of  the  image  stored  in  the  core  or  disk  memory  that  is  to  be 
processed.  Setting  bit  3  of  the  switch  register  selects  a  128-  by 
128-element  image,  setting  bit  4,  a  256-  by  256-element  image  and 
setting  bit  5,  a  512-  by  512-element  image.  Images  containing  128  by 
128  elements  or  256  by  256  elements  are  stored  in  the  65  kword  data 
memory  of  the  array  transform  processor,  while  images  containing  512  by 
512  elements  are  stored  in  one  of  4  files  in  the  disk  memory.  For  128- 
by  128-element  images,  the  user  must  reply  to  a  question  on  the  teletype 
to  specify  which  of  the  4  images  in  the  array  processor  data  memory  he 
wishes  to  process.  Similarly,  the  user  must  specify  one  of  the  4  disk 
files  when  he  is  processing  512-  by  512-element  images. 

The  television  monitor  displays  a  total  area  of  512  by  512 
elements.  A  user  wishing  to  view  simultaneously  several  smaller  images 
on  the  display  rather  than  a  single  one  containing  512  by  512  elements 
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selects  the  desired  display  size  by  using  the  switch  register.  The  area 
that  the  image  will  occupy  on  the  display  is  set  to  128  by  128  by 
setting  bit  6  of  the  switch  register,  to  256  by  256  by  setting  bit  7, 
and  to  512  by  512  by  setting  bit  8.  The  size  of  the  display  area  need 
not  be  equal  to  the  size  of  the  working  image.  For  example,  setting 
bits  3  and  7  will  cause  a  128-  by  128-element  image  to  be  displayed  on  a 
256-  by  256-element  display  area,  whereas  setting  bits  3  and  8  will 
cause  the  128-  by  128-element  image  to  fill  the  complete  display  area. 
If  bit  9  of  the  switch  register  is  set  and  the  display  area  is  128  by 
128  or  256  by  256,  then  the  new  image  is  stored  in  the  display  position 
one  beyond  the  last  stored  image.  When  the  display  area  is  full,  or  if 
the  selected  display  area  is  512  by  512,  then  the  display  is  erased 
before  the  new  image  is  written. 

When  the  desired  options  have  been  set  on  the  switch  register, 
image-processing  commands  can  be  given  interactively  on  the  teletype. 
The  system  types  an  asterisk  (*)  to  indicate  that  it  is  ready  to  accept 
a  command  from  the  user.  These  commands  are  in  the  form  of  integer 
numbers  containing  4  digits  or  less.  In  some  cases  ,  such  as  the 
2-dimensional  FFT  of  a  256-  by  256-element  image,  the  system  will 
immediately  execute  the  operation  and  return  to  the  command  mode.  In 
other  cases,  such  as  filtering  operations  that  require  parameter 
specification,  the  user  must  reply  to  questions  on  the  teletype  before 
the  processing  will  proceed. 

The  first  digit  of  each  integer  command  number  describes  the 
general  class  of  the  operation.  There  are  currently  9  classes  of 
operations  corresponding  to  the  first  digits  1  to  9: 

1  =  display  operations 

2  =  input/output  operations 

3  =  FFT  operations 
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4  =  gray  scale  normalization 

5  =  mathematical  operations 

6  =  frequency-domain  filters 

7  =  pattern  generation 

8  =  geometrical  operations 

9  =  miscellaneous  operations 

For  example,  any  command  that  begins  with  a  1  is  a  display  operation, 
whereas  any  command  that  begins  with  a  6  is  a  frequency-domain  filtering 
operation.  Hie  remaining  digits  of  the  command  number  define  the 
specific  operation  in  each  class.  For  example,  in  the  class  of  display 
operations,  command  "11"  displays  an  image  on  the  television  monitor 
while  command  "16"  graphs  a  selected  row  of  an  image.  In  the  class  of 
frequency-domain  filtering  operations,  command  "612"  multiplies  the 
working  image  (which  in  this  case  must  be  a  2-dimensional  FFT)  by  the 
filter  required  to  restore  a  particular  type  of  focus  blurring.  A 
listing  of  the  available  commands  is  given  in  Appendix  B. 

In  addition  to  the  working  image,  two  512-word  integer  buffers 
are  accessible  to  the  user.  The  "IZT"  buffer  contains  the  display 
transfer  function,  i.e.  the  function  that  maps  the  stored  gray  scale 
values  to  displayed  gray  levels.  This  transfer  function  is  specified 
with  command  "99".  Other  operations,  such  as  the  geometrical 
transformation  specified  by  command  "85"  and  the  frequency-domain  filter 
specified  by  command  "661",  also  use  the  function  stored  in  this  buffer. 
The  second  buffer  ("IZ")  is  used  for  temporary  storage  of  information, 
such  as  the  gray  scale  histogram  calculated  with  command  "55".  This 
buffer  is  also  used  as  a  working  area  and  most  processing  operations 
will  alter  its  contents. 
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APPENDIX  B 


Listing  of  Interactive  Commands 


When  the  system  prompts  by  typing  an  asterisk  (*)  on  the 
teletype,  then  it  is  ready  to  receive  one  of  the  following  commands: 
********************************** 

*****  pep  SCAN  CONVERTER  DISPLAV  OPERATIONS  ***** 


COMMAND 

OPERATION 

10 

CLEAR  TV  DISPLAV 

11 

DISPLAV  AN  IMAGE 

12 

DISPLAV  A  SERIES 

OF  IMAGES  ON  DIS 

15 

GRAPH  A  BUFFER 

16 

GRAPH  A  ROW 

171 

LIST  A  BUFFER  ON 

LP 

172 

LIST  A  REGION  ON 

TTV 

173 

LIST  A  REGION  ON 

LP 

********************************** 

*****  INPUT/OUTPUT  operations  ***** 

COMMAND  OPERATION 

20  READ  IMAGE  ON  MT 

201  READ  AND  REFORMAT  N  BY  M  IMAGE  ON  MT 

21  WRITE  IMAGE  ON  MT 

22  REWIND  MT 

231  TV  DIGITIZE  <  CORRECT  FOR  4:3  ASPECT) 

232  TV  DIGITIZE  <512X480,  NO  CORRECTION) 

233  DIGITIZE  POINT  ON  TV  AS  SET  BV  TRACKBALL 

241  MOVE  128  IMAGE  IN  A.  P 

242  MOVE  512  IMAGE  ON  DK2 

25  READ  IMAGE  ON  DK2 

26  WRITE  IMAGE  ON  DK2 

271  READ  G.  S.  TRANSFER  FUNCTION  ON  DK2 

272  WRITE  G.  S.  TRANSFER  FUNCTION  ON  DK2 

273  READ  G  S.  HISTOGRAM  ON  Dk.2 

274  WRITE  G  S.  HISTOGRAM  ON  DK2 

275  MOVE  IZT  BUFFER  TO  P  BUFFER 

276  MOVE  P  BUFFER  TO  IZT  BUFFER 

277  CONVERT  P  BUFFER  TO  INV  WIENER  FILTER 

278  GENERATE  HALLS  MODEL  FOR  HVS  IN  P  BUFFER 
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ft*******************  *:*:**:  **:  *4:*. 

*****  FFT  OPERATIONS  ***** 

COMMAND  OPERATION 

30  2D  FFT 

31  INVERSE  2D  FFT 

321  MULTIPLY  ONE  FFT  BY  ANOTHER 

322  MULTIPLY  ONE  FFT  BY  CC  OF  ANOTHER 

35  MODULUS  OF  AN  FFT 

36  CALCULATE  1-D  AMPLITUDE  SPECTRUM  OF  IMAGE 

3?  CONVERT  FFT  TO  INVERSE  WIENER  FILTER 

*********************************:* 

*****  GRAYSCALE  NORMALIZATION  ***** 

COMMAND  OPERATION 

401  NORMALIZE  <3276 7  MAX) 

402  NORMALIZE  <0  TO  32767) 

411  CLIP  <  G.  S.  UNITS) 

412  CLIP  (UNITS  OF  ST  DEVS) 

413  CLIP  AND  NORM  <G.  S.  UNITS) 

414  CLIP  AND  NORM  (UNITS  OF  ST  DEVS) 


♦  +  ************************1******** 

*****  math  operations  ***** 

COMMAND  OPERATION 

501  LOGARITHM 

502  EXPONENTIATE 

511  ADD  ONE  IMAGE  TO  ANOTHER 

512  SUBTRACT  ONE  IMAGE  FROM  ANOTHER 

513  MULTIPLY  ONE  IMAGE  BY  ANOTHER 

514  DIVIDE  ONE  IMAGE  BY  ANOTHER 

521  ADD  A  CONSTANT  TO  A  REGION 

522  MULTIPLY  A  REGION  BY  A  CONSTANT 

53  TRANSFORM  USING  G.  S.  TRANSFER  FN 

54  '  COMPLIMENT  GRAYSCALE 

55  G  S  HISTOGRAM  < I Z  BUFFER) 

561  CALCULATE  MIN  AND  MAX  VALUES 

562  CALCULATE  MEAN  AND  RMS  VALUES 

571  2X2  EDGE  DETECTOR  (SQR) 

572  2X2  EDGE  DETECTOR  (ABS) 

58  DIRECTION  OF  STRUCTURES  TO  GRAY  LEVEL 

591  MAD  CORRELATION 

592  MATHEMATICAL  CORRELATION 

593  SSDA  CORRELATION 
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******************************** 
*****  FREQUENCY-DOMAIN  FILTERS  ***** 


COMMAND  OPERATION 


601 

MOTION  DEGRADE 

602 

MOTION  RESTORE 

611 

FOCUS  DEGRADE 

612 

FOCUS  RESTORE 

621 

HIGH-PASS:  F(U)=SQRT(W) 

622 

HIGH-PASS:  F(W)=W 

623 

HIGH-PASS.  F(W)=W**2 

641 

LOW-PASS  (EXP  CUTOFF) 

642 

HIGH-PASS  (EXP  CUTOFF) 

661 

FILTER  WITH  LOOKUP  TABLE 

IN 

IZT  BUFFER 

662 

FILTER  WITH  LOOKUP  TABLE 

IN 

P  BUFFER 

631 

H VS  LOW-PASS  (HALLS  MODEL) 

682 

H VS  HIGH-PASS  (HALLS  MODEL) 

********************************** 

*****  PATTERN  generation  ***** 

COMMAND  OPERATION 

711  DRAW  A  GRAYSCALE  CHART 

712  DRAW  G. S.  CHARTS  TO  LEFT  OF  IMAGE 

721  DRAW  A  GRID 

722  DRAW  A  BOX  AROUND  A  REGION 

723  DRAW  DIVIDING  LINES  BETWEEN  IMAGES 

724  DRAW  A  CROSS  AT  HIGHEST  POINT 

731  GENERATE  A  GRAYSCALE  CHART 

732  GENERATE  A  PATTERN  OF  SQUARES 

733  GENERATE  A  PATTERN  OF  RANDOM  NOISE 

734  GENERATE  A  CONSTANT  LEVEL 

77  SET  IMAGE  TO  ZERO 

********************************** 

*****  geometrical  operations  ***** 

COMMAND  OPERATION 

801  MAGNIFY  A  REGION 

802  MINIFY  A  REGION 

803  READ  128  REGION  OF  512  IMAGE 

804  READ  256  REGION  OF  512  IMAGE 

805  READ  SPECIFIED  REGION  OF  512  IMAGE 

306  WRITE  SPECIFIED  REGION  OF  512  IMAGE 

82  TRANSPOSE 

83  REVERSE  X  AXIS 

341  SCROLL  IMAGE  HORIZONTALLY 

842  SCROLL  IMAGE  VERTICALLY 

85  GEOMETRICAL  TRANSFORM  <T  F  IN  IZT  BUFFER) 
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********************************** 

*****  MISC  OPERATIONS  ***** 

COMMAND  OPERATION 

900  INTRODUCTORY  INFO  ON  SYSTEM  COMMANDS 

901  INFO  ON  DISPLAY  COMMANDS  (  THESE  START  WITH  1) 

902  INFO  ON  I/O  COMMANDS  (THESE  START  WITH  2) 

909  INFO  ON  MISC  COMMANDS  (THESE  START  WITH  9) 

93  MATCH  OPPOSITE  EDGES  OF  IMAGE 

94  RADIAL  DIST  FN  (P  BUFFER) 

99  GENERATE  G.  S.  TRANSFER  FN 

999  EXIT  TO  POP  11/40  MONITOR 


********************************** 
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